39 research outputs found
DisDiff: Unsupervised Disentanglement of Diffusion Probabilistic Models
Targeting to understand the underlying explainable factors behind
observations and modeling the conditional generation process on these factors,
we connect disentangled representation learning to Diffusion Probabilistic
Models (DPMs) to take advantage of the remarkable modeling ability of DPMs. We
propose a new task, disentanglement of (DPMs): given a pre-trained DPM, without
any annotations of the factors, the task is to automatically discover the
inherent factors behind the observations and disentangle the gradient fields of
DPM into sub-gradient fields, each conditioned on the representation of each
discovered factor. With disentangled DPMs, those inherent factors can be
automatically discovered, explicitly represented, and clearly injected into the
diffusion process via the sub-gradient fields. To tackle this task, we devise
an unsupervised approach named DisDiff, achieving disentangled representation
learning in the framework of DPMs. Extensive experiments on synthetic and
real-world datasets demonstrate the effectiveness of DisDiff.Comment: Accepted by NeurIPS 202
Vector-based Representation is the Key: A Study on Disentanglement and Compositional Generalization
Recognizing elementary underlying concepts from observations
(disentanglement) and generating novel combinations of these concepts
(compositional generalization) are fundamental abilities for humans to support
rapid knowledge learning and generalize to new tasks, with which the deep
learning models struggle. Towards human-like intelligence, various works on
disentangled representation learning have been proposed, and recently some
studies on compositional generalization have been presented. However, few works
study the relationship between disentanglement and compositional
generalization, and the observed results are inconsistent. In this paper, we
study several typical disentangled representation learning works in terms of
both disentanglement and compositional generalization abilities, and we provide
an important insight: vector-based representation (using a vector instead of a
scalar to represent a concept) is the key to empower both good disentanglement
and strong compositional generalization. This insight also resonates the
neuroscience research that the brain encodes information in neuron population
activity rather than individual neurons. Motivated by this observation, we
further propose a method to reform the scalar-based disentanglement works
(-TCVAE and FactorVAE) to be vector-based to increase both capabilities.
We investigate the impact of the dimensions of vector-based representation and
one important question: whether better disentanglement indicates higher
compositional generalization. In summary, our study demonstrates that it is
possible to achieve both good concept recognition and novel concept
composition, contributing an important step towards human-like intelligence.Comment: Preprin
Health-related effects and improving extractability of cereal arabinoxylans
Arabinoxylans (AXs) are major dietary fibers. They are composed of backbone chains of -(1â4)- linked xylose residues to which -l-arabinose are linked in the second and/or third carbon positions. Recently, AXs have attracted a great deal of attention because of their biological activities such as their immunomodulatory potential. Extraction of AXs has some difficulties; therefore, various methods have beenusedto increase the extractability ofAXs withvaryingdegrees of success, suchas alkaline, enzymatic, mechanical extraction. However, some of these treatments have been reported to be either expensive, such as enzymatic treatments, or produce hazardous wastes and are non-environmentally friendly, such as alkaline treatments. On the other hand, mechanical assisted extraction, especially extrusion cooking, is an innovative pre-treatment that has been used to increase the solubility of AXs. The aim of the current review article is to point out the health-related effects and to discuss the current research on the extraction methods of AXs
An Adaptive Data Gathering Scheme for Multi-Hop Wireless Sensor Networks Based on Compressed Sensing and Network Coding
Joint design of compressed sensing (CS) and network coding (NC) has been demonstrated to provide a new data gathering paradigm for multi-hop wireless sensor networks (WSNs). By exploiting the correlation of the network sensed data, a variety of data gathering schemes based on NC and CS (Compressed Data GatheringâCDG) have been proposed. However, these schemes assume that the sparsity of the network sensed data is constant and the value of the sparsity is known before starting each data gathering epoch, thus they ignore the variation of the data observed by the WSNs which are deployed in practical circumstances. In this paper, we present a complete design of the feedback CDG scheme where the sink node adaptively queries those interested nodes to acquire an appropriate number of measurements. The adaptive measurement-formation procedure and its termination rules are proposed and analyzed in detail. Moreover, in order to minimize the number of overall transmissions in the formation procedure of each measurement, we have developed a NP-complete model (Maximum Leaf Nodes Minimum Steiner NodesâMLMS) and realized a scalable greedy algorithm to solve the problem. Experimental results show that the proposed measurement-formation method outperforms previous schemes, and experiments on both datasets from ocean temperature and practical network deployment also prove the effectiveness of our proposed feedback CDG scheme